How to manipulate a gene list
The golden rule before we start Always keep a backup of the output from the pipeline, never work on the original data!
There are various tasks you might wish to perform on a gene list
- Search for our favourite gene
- Sort / Rank according to statistic or p-value
- Filter to obtain genes with particular cut-off and fold-change
- Re-order / re-arrange columns
Our recommended tools to perform such operations would be a programming language such as R / Python so that the operations can be scripted and automated. As a compromise, our Galaxy server provides various operations through an intuitive interface and allows the user to build a workflow to a chain of simple operations.
Different online tools or GUIs will take different types of input, but will probably involve some combination of the operations mentioned above. Therefore, we will introduce some tools in Galaxy that should allow you to manipulate your data into the required form. Later, we will give examples of workflows for particular gene set enrichment / pathways analysis tools.
In the first generic example, we will show how to filter a gene list using Galaxy:-
How to apply an alternative filter to the gene list in Galaxy
Convert to tabular
Before we can go further, Galaxy needs to convert the data you just uploaded into tabular form by replacing commas with a tab. This can be done using the Text Manipulation -> Convert delimiters to TAB option. Make sure that Commas is selected from the drop-down.
Filter
We can apply filtering using the menu option Filter and Sort -> Filter data on any column using simple expressions. Here we use the condition c7 <0.01 and c3 > 1.5 to require that Column 7 (adjusted p-value) is less than 0.05 and Column 3 (log fold-change) is greater than 1.5.
You could also use this menu to filter on genes on a particular chromosome or within a certain range of start and end positions
Sorting by a column
We can also sort the table by values in a particular column. e.g. the log\(_2\) fold change. 
- can also sort by multiple columns
- e.g. Chromosome ## R workflow
For those keen on R, equivalent operations can be performed with the dplyr package. See our intermediate R course for details.
library(dplyr)
deTable <- read.csv("t47d_Treatment_DEA_Prog-vs-Control_all.csv")
filteredTable <- filter(deTable, padj < 0.05, log2FoldChange > 1.5)
sortedTable <- arrange(filteredTable, log2FoldChange)
reducedTable <- select(filteredTable, X,baseMean,log2FoldChange)
Gene-Ontologies and Pathways
Gene-Ontology Analysis
In the early days of microarray analysis, people were happy if they got a handful of differentially-expressed genes that they could validate or follow-up. However, with later technologies (and depending on the experimental setup) we might have thousands of statistically-significant results, which no-one has the time to follow-up. Also, we might be interested in pathways / mechanisms that are altered and not just individual genes.
In this section we move towards discovering if our results are biologically significant. Are the genes that we have picked statistical flukes, or are there some commonalities.
There are two different approaches one might use, and we will cover the theory behind both
Theory Part I: Over-representation analysis
- “Threshold-based”: require defintion of a statistical threshold to define list of genes to test (e.g. FDR < 0.01)
- Hypergeometric test or Fisher’s Exact test generally used.
The question we are asking here is;
“Are the number of DE genes associated with Theme X significantly greater than what we might expect by chance alone?”
Where Theme X could be genes belonging to a particular GO (Gene Onotology) term or pathway.
Let’s imagine that we have a bag full of balls. Each balls represents a gene in the gene universe. - Paint the balls representing our selected list grey, and paint the rest red.
In this small example, we can know in advance the total number of balls and total number in our category of interest
- Total number of balls: 40
- Total number of interesting (grey) balls: 10
Now, lets select a number (say, 12) of the balls at random without seeing into the bag and look at what we get
We can express a particular choice of balls as a table:-
| Grey Selected |
8 |
2 |
10 |
| Grey Not Selected |
4 |
26 |
30 |
| Total |
12 |
6 |
40 |
The formula for Fishers exact test is;
\[ p = \frac{\binom{a + b}{a}\binom{c +d}{c}}{\binom{n}{a +c}} = \frac{(a+b)!(c+d)!(a+c)!(b+d)!}{a!b!c!d!n!} \] with :-
| Grey Selected |
a |
b |
a + b |
| Grey Not Selected |
c |
d |
c + d |
| Total |
a + c |
b +d |
a + b + c + d (=n) |
or less formally;
P = (ways of choosing grey balls) X (ways of non-grey balls amongst subset) / ways of choosing subset
Fortunately, we have software to calculate these quantities!
Software for conducting a over-representation test (goseq)
We will be using goseq, which is a software package available through Bioconductor. However, rather having to write R code, we will be using the package through our institute’s Galaxy server.
This package has been specifically-developed for use with RNA-seq data. Plenty of methods have been applied to microarray data, but the assumptions might not hold for RNA-seq. For instance, goseq will adjust for the length of a gene.
Preparing the data for an over-representation test
We can start with the list that contains results for all genes; Let’s take t47d_Treatment_DEA_Prog-vs-Control_all.csv
deTable <- read.csv("t47d_Treatment_DEA_Prog-vs-Control_all.csv")
deTable
The goal is to obtain a table with two columns; the first containing the gene identifier, and the second being TRUE or FALSE indicating whether the gene is differentially-expressed at a given cut-off (e.g. 0.05).
There are several ways of doing this. If you wish to see the R code (using dplyr), click the ‘Code’ button on the right.
library(dplyr)
deTable <- read.csv("t47d_Treatment_DEA_Prog-vs-Control_all.csv")
mutate(deTable, DE = padj < 0.05) %>%
mutate(DE = ifelse(is.na(DE),FALSE,DE)) %>%
select(X, DE) -> filteredTable
filteredTable
write.table(filteredTable,"de-table-for-goseq.txt",quote=FALSE,row.names=FALSE)
Manipulating the file can also be done in Galaxy.
Importing the Gene list into Galaxy
Import the csv file
Get Data -> Upload Data
Retain columns of interest
Put c1,c2,c3,c4,c5,c6,c7,c12 to retain columns 1,2,3,4,5,6,7 and 12
Text Manipulation -> Cut columns from a table
The cleaned dataset
The output from the previous step will be referred to as our cleaned dataset
Theory Part II: Threshold-free
For these tests, we don’t have to specify a statistical threshold and use the test statistics from all genes as the input to the test. The popular Gene Set Enrichment Analysis (GSEA) uses this approach. These tests can be used to detect differential expression for a group of genes, even when the effects are too small or there is too little data to detect the genes individually.
- Rank all your genes according to test statistic or fold-change
- For a particular gene set of interest, look down the gene list and compute a score
- Go up one unit on the y-axis if the gene you encounter is contained in your gene list
- Go down one unit if the gene is not contained in the gene list
- Plot the score against position in gene list
- If there is a peak at the left or right, then your set is enriched.
The Broad institute provides a version of GSEA that can be run via a java application. The required input text contains a set of gene identifiers that are ordered according to some test statistic.
library(dplyr)
deTable <- read.csv("deseq2_results/t47d_Treatment_DEA_Prog-vs-Control_all.csv")
deTable %>% arrange(desc(stat)) %>%
dplyr:::select(symbol, stat) %>%
filter(!is.na(symbol)) -> orderedTable
orderedTable
Preparing the gene set for a GSEA analysis
Ranking the genes
From the cleaned table created above, sort on column 5 (the test statistic)
Filter and Sort -> Sort data in ascending or descending order
Giving the output:-
Cutting the columns
Extract columns 8 (gene symbol) followed by test statistic; c8,c5
Text Manipulation -> Cut columns from a table
LS0tCnRpdGxlOiAiQmV5b25kIHRoZSBHZW5lIExpc3QiCmF1dGhvcjogIk1hcmsgRHVubmluZzsgbWFyayAnZG90JyBkdW5uaW5nICdhdCcgY3J1ay5jYW0uYWMudWsiCmRhdGU6ICdgciBmb3JtYXQoU3lzLnRpbWUoKSwgIkxhc3QgbW9kaWZpZWQ6ICVkICViICVZIilgJwpvdXRwdXQ6CiAgaHRtbF9ub3RlYm9vazoKICAgIHRvYzogeWVzCiAgICB0b2NfZmxvYXQ6IHllcwogIGh0bWxfZG9jdW1lbnQ6CiAgICB0b2M6IHllcwotLS0KCmBgYHtyIHNldHVwLCBpbmNsdWRlPUZBTFNFLGVjaG89RkFMU0V9CmtuaXRyOjpvcHRzX2NodW5rJHNldChlY2hvID0gRkFMU0UpCmBgYAoKIyBIb3cgdG8gbWFuaXB1bGF0ZSBhIGdlbmUgbGlzdAoKVGhlIGdvbGRlbiBydWxlIGJlZm9yZSB3ZSBzdGFydCAqKipBbHdheXMga2VlcCBhIGJhY2t1cCBvZiB0aGUgb3V0cHV0IGZyb20gdGhlIHBpcGVsaW5lLCBuZXZlciB3b3JrIG9uIHRoZSBvcmlnaW5hbCBkYXRhKioqIQoKVGhlcmUgYXJlIHZhcmlvdXMgdGFza3MgeW91IG1pZ2h0IHdpc2ggdG8gcGVyZm9ybSBvbiBhIGdlbmUgbGlzdAoKLSBTZWFyY2ggZm9yIG91ciBmYXZvdXJpdGUgZ2VuZQotIFNvcnQgLyBSYW5rIGFjY29yZGluZyB0byBzdGF0aXN0aWMgb3IgcC12YWx1ZQotIEZpbHRlciB0byBvYnRhaW4gZ2VuZXMgd2l0aCBwYXJ0aWN1bGFyIGN1dC1vZmYgYW5kIGZvbGQtY2hhbmdlCi0gUmUtb3JkZXIgLyByZS1hcnJhbmdlIGNvbHVtbnMKCk91ciByZWNvbW1lbmRlZCB0b29scyB0byBwZXJmb3JtIHN1Y2ggb3BlcmF0aW9ucyB3b3VsZCBiZSBhIHByb2dyYW1taW5nIGxhbmd1YWdlIHN1Y2ggYXMgUiAvIFB5dGhvbiBzbyB0aGF0IHRoZSBvcGVyYXRpb25zIGNhbiBiZSBzY3JpcHRlZCBhbmQgYXV0b21hdGVkLiBBcyBhIGNvbXByb21pc2UsIG91ciBHYWxheHkgc2VydmVyIHByb3ZpZGVzIHZhcmlvdXMgb3BlcmF0aW9ucyB0aHJvdWdoIGFuIGludHVpdGl2ZSBpbnRlcmZhY2UgYW5kIGFsbG93cyB0aGUgdXNlciB0byBidWlsZCBhIHdvcmtmbG93IHRvIGEgY2hhaW4gb2Ygc2ltcGxlIG9wZXJhdGlvbnMuCgpEaWZmZXJlbnQgb25saW5lIHRvb2xzIG9yIEdVSXMgd2lsbCB0YWtlIGRpZmZlcmVudCB0eXBlcyBvZiBpbnB1dCwgYnV0IHdpbGwgcHJvYmFibHkgaW52b2x2ZSBzb21lIGNvbWJpbmF0aW9uIG9mIHRoZSBvcGVyYXRpb25zIG1lbnRpb25lZCBhYm92ZS4gVGhlcmVmb3JlLCB3ZSB3aWxsIGludHJvZHVjZSBzb21lIHRvb2xzIGluIEdhbGF4eSB0aGF0IHNob3VsZCBhbGxvdyB5b3UgdG8gbWFuaXB1bGF0ZSB5b3VyIGRhdGEgaW50byB0aGUgcmVxdWlyZWQgZm9ybS4gTGF0ZXIsIHdlIHdpbGwgZ2l2ZSBleGFtcGxlcyBvZiB3b3JrZmxvd3MgZm9yIHBhcnRpY3VsYXIgZ2VuZSBzZXQgZW5yaWNobWVudCAvIHBhdGh3YXlzIGFuYWx5c2lzIHRvb2xzLgoKSW4gdGhlIGZpcnN0IGdlbmVyaWMgZXhhbXBsZSwgd2Ugd2lsbCBzaG93IGhvdyB0byBmaWx0ZXIgYSBnZW5lIGxpc3QgdXNpbmcgR2FsYXh5Oi0KCiMjIEhvdyB0byBhcHBseSBhbiBhbHRlcm5hdGl2ZSBmaWx0ZXIgdG8gdGhlIGdlbmUgbGlzdCBpbiBHYWxheHkKCiMjIyBVcGxvYWQgdGhlIGdlbmUgbGlzdCBpbiBjc3YgZm9ybQoKVGhlIGdlbmUgbGlzdCBgLmNzdmAgZmlsZSBjYW4gYmUgdXBsb2FkZWQgaW50byBHYWxheHkuIEluIHRoaXMgZXhhbXBsZSB3ZSB3YW50IHRvIGNob29zZSB0aGUgZmlsZSB3aXRoIG5vIHAtdmFsdWUgY3V0LW9mZnMgYXBwbGllZAoKIVtdKGltYWdlcy9maWx0ZXIxLnBuZykKCiMjIyBDb252ZXJ0IHRvIHRhYnVsYXIKCkJlZm9yZSB3ZSBjYW4gZ28gZnVydGhlciwgR2FsYXh5IG5lZWRzIHRvIGNvbnZlcnQgdGhlIGRhdGEgeW91IGp1c3QgdXBsb2FkZWQgaW50byAqdGFidWxhciogZm9ybSBieSByZXBsYWNpbmcgY29tbWFzIHdpdGggYSB0YWIuIFRoaXMgY2FuIGJlIGRvbmUgdXNpbmcgdGhlICoqKlRleHQgTWFuaXB1bGF0aW9uKioqIC0+ICoqKkNvbnZlcnQgZGVsaW1pdGVycyB0byBUQUIqKiogb3B0aW9uLiBNYWtlIHN1cmUgdGhhdCAqQ29tbWFzKiBpcyBzZWxlY3RlZCBmcm9tIHRoZSBkcm9wLWRvd24uCgohW10oaW1hZ2VzL2ZpbHRlcjIucG5nKQoKIyMjIEZpbHRlcgoKV2UgY2FuIGFwcGx5IGZpbHRlcmluZyB1c2luZyB0aGUgbWVudSBvcHRpb24gKioqRmlsdGVyIGFuZCBTb3J0KioqIC0+ICoqKkZpbHRlciBkYXRhIG9uIGFueSBjb2x1bW4gdXNpbmcgc2ltcGxlIGV4cHJlc3Npb25zKioqLiBIZXJlIHdlIHVzZSB0aGUgY29uZGl0aW9uIGBjNyA8MC4wMSBhbmQgYzMgPiAxLjVgIHRvIHJlcXVpcmUgdGhhdCBDb2x1bW4gNyAoYWRqdXN0ZWQgcC12YWx1ZSkgaXMgbGVzcyB0aGFuIDAuMDUgKmFuZCogQ29sdW1uIDMgKGxvZyBmb2xkLWNoYW5nZSkgaXMgZ3JlYXRlciB0aGFuIDEuNS4KCgohW10oaW1hZ2VzL2ZpbHRlcjMucG5nKQoKWW91IGNvdWxkIGFsc28gdXNlIHRoaXMgbWVudSB0byBmaWx0ZXIgb24gZ2VuZXMgb24gYSBwYXJ0aWN1bGFyIGNocm9tb3NvbWUgb3Igd2l0aGluIGEgY2VydGFpbiByYW5nZSBvZiBzdGFydCBhbmQgZW5kIHBvc2l0aW9ucwoKIyMgU2VsZWN0aW5nIGNvbHVtbnMKCiFbXShpbWFnZXMvZmlsdGVyNC5wbmcpCgojIyBTb3J0aW5nIGJ5IGEgY29sdW1uCgpXZSBjYW4gYWxzbyBzb3J0IHRoZSB0YWJsZSBieSB2YWx1ZXMgaW4gYSBwYXJ0aWN1bGFyIGNvbHVtbi4gZS5nLiB0aGUgbG9nJF8yJCBmb2xkIGNoYW5nZS4KIVtdKGltYWdlcy9maWx0ZXI1LnBuZykKCi0gY2FuIGFsc28gc29ydCBieSBtdWx0aXBsZSBjb2x1bW5zCiAgICArIGUuZy4gQ2hyb21vc29tZSAKIyMgUiB3b3JrZmxvdwoKRm9yIHRob3NlIGtlZW4gb24gUiwgZXF1aXZhbGVudCBvcGVyYXRpb25zIGNhbiBiZSBwZXJmb3JtZWQgd2l0aCB0aGUgYGRwbHlyYCBwYWNrYWdlLiBTZWUgb3VyIFtpbnRlcm1lZGlhdGUgUl0oaHR0cDovL2Jpb2luZm9ybWF0aWNzLWNvcmUtc2hhcmVkLXRyYWluaW5nLmdpdGh1Yi5pby9yLWludGVybWVkaWF0ZS8pIGNvdXJzZSBmb3IgZGV0YWlscy4gCgpgYGB7ciBtZXNzYWdlPUZBTFNFfQpsaWJyYXJ5KGRwbHlyKQpkZVRhYmxlIDwtIHJlYWQuY3N2KCJ0NDdkX1RyZWF0bWVudF9ERUFfUHJvZy12cy1Db250cm9sX2FsbC5jc3YiKQpmaWx0ZXJlZFRhYmxlIDwtIGZpbHRlcihkZVRhYmxlLCBwYWRqIDwgMC4wNSwgbG9nMkZvbGRDaGFuZ2UgPiAxLjUpCnNvcnRlZFRhYmxlIDwtIGFycmFuZ2UoZmlsdGVyZWRUYWJsZSwgbG9nMkZvbGRDaGFuZ2UpCnJlZHVjZWRUYWJsZSA8LSBzZWxlY3QoZmlsdGVyZWRUYWJsZSwgWCxiYXNlTWVhbixsb2cyRm9sZENoYW5nZSkKYGBgCgojIEdlbmUtT250b2xvZ2llcyBhbmQgUGF0aHdheXMKCiMjIEdlbmUtT250b2xvZ3kgQW5hbHlzaXMKCkluIHRoZSBlYXJseSBkYXlzIG9mIG1pY3JvYXJyYXkgYW5hbHlzaXMsIHBlb3BsZSB3ZXJlIGhhcHB5IGlmIHRoZXkgZ290IGEgaGFuZGZ1bCBvZiBkaWZmZXJlbnRpYWxseS1leHByZXNzZWQgZ2VuZXMgdGhhdCB0aGV5IGNvdWxkIHZhbGlkYXRlIG9yIGZvbGxvdy11cC4gSG93ZXZlciwgd2l0aCBsYXRlciB0ZWNobm9sb2dpZXMgKGFuZCBkZXBlbmRpbmcgb24gdGhlIGV4cGVyaW1lbnRhbCBzZXR1cCkgd2UgbWlnaHQgaGF2ZSB0aG91c2FuZHMgb2Ygc3RhdGlzdGljYWxseS1zaWduaWZpY2FudCByZXN1bHRzLCB3aGljaCBuby1vbmUgaGFzIHRoZSB0aW1lIHRvIGZvbGxvdy11cC4gQWxzbywgd2UgbWlnaHQgYmUgaW50ZXJlc3RlZCBpbiBwYXRod2F5cyAvIG1lY2hhbmlzbXMgdGhhdCBhcmUgYWx0ZXJlZCBhbmQgbm90IGp1c3QgaW5kaXZpZHVhbCBnZW5lcy4KCkluIHRoaXMgc2VjdGlvbiB3ZSBtb3ZlIHRvd2FyZHMgZGlzY292ZXJpbmcgaWYgb3VyIHJlc3VsdHMgYXJlICoqKmJpb2xvZ2ljYWxseSBzaWduaWZpY2FudCoqKi4gQXJlIHRoZSBnZW5lcyB0aGF0IHdlIGhhdmUgcGlja2VkIHN0YXRpc3RpY2FsIGZsdWtlcywgb3IgYXJlIHRoZXJlIHNvbWUgY29tbW9uYWxpdGllcy4gCgpUaGVyZSBhcmUgdHdvIGRpZmZlcmVudCBhcHByb2FjaGVzIG9uZSBtaWdodCB1c2UsIGFuZCB3ZSB3aWxsIGNvdmVyIHRoZSB0aGVvcnkgYmVoaW5kIGJvdGgKCgojIyBUaGVvcnkgUGFydCBJOiBPdmVyLXJlcHJlc2VudGF0aW9uIGFuYWx5c2lzCgotICJUaHJlc2hvbGQtYmFzZWQiOiByZXF1aXJlIGRlZmludGlvbiBvZiBhIHN0YXRpc3RpY2FsIHRocmVzaG9sZCB0byBkZWZpbmUgbGlzdCBvZiBnZW5lcyB0byB0ZXN0IChlLmcuIEZEUiA8IDAuMDEpCi0gSHlwZXJnZW9tZXRyaWMgdGVzdCBvciBGaXNoZXIncyBFeGFjdCB0ZXN0IGdlbmVyYWxseSB1c2VkLgoKVGhlIHF1ZXN0aW9uIHdlIGFyZSBhc2tpbmcgaGVyZSBpczsKCj4gKioqIkFyZSB0aGUgbnVtYmVyIG9mIERFIGdlbmVzIGFzc29jaWF0ZWQgd2l0aCBUaGVtZSBYIHNpZ25pZmljYW50bHkgZ3JlYXRlciB0aGFuIHdoYXQgd2UgbWlnaHQgZXhwZWN0IGJ5IGNoYW5jZSBhbG9uZT8iKioqCgpXaGVyZSBUaGVtZSBYIGNvdWxkIGJlIGdlbmVzIGJlbG9uZ2luZyB0byBhIHBhcnRpY3VsYXIgR08gKEdlbmUgT25vdG9sb2d5KSB0ZXJtIG9yIHBhdGh3YXkuCgpMZXQncyBpbWFnaW5lIHRoYXQgd2UgaGF2ZSBhIGJhZyBmdWxsIG9mIGJhbGxzLiBFYWNoIGJhbGxzIHJlcHJlc2VudHMgYSBnZW5lIGluIHRoZSAqZ2VuZSB1bml2ZXJzZSouIAotIFBhaW50IHRoZSBiYWxscyByZXByZXNlbnRpbmcgb3VyIHNlbGVjdGVkIGxpc3QgZ3JleSwgYW5kIHBhaW50IHRoZSByZXN0IHJlZC4KCgohW10oaW1hZ2VzL2JhZy1hbmQtYmFsbHMucG5nKQoKSW4gdGhpcyBzbWFsbCBleGFtcGxlLCB3ZSBjYW4ga25vdyBpbiBhZHZhbmNlIHRoZSB0b3RhbCBudW1iZXIgb2YgYmFsbHMgYW5kIHRvdGFsIG51bWJlciBpbiBvdXIgY2F0ZWdvcnkgb2YgaW50ZXJlc3QKCi0gVG90YWwgbnVtYmVyIG9mIGJhbGxzOiA0MAotIFRvdGFsIG51bWJlciBvZiBpbnRlcmVzdGluZyAoZ3JleSkgYmFsbHM6IDEwCgpOb3csIGxldHMgc2VsZWN0IGEgbnVtYmVyIChzYXksIDEyKSBvZiB0aGUgYmFsbHMgYXQgcmFuZG9tIHdpdGhvdXQgc2VlaW5nIGludG8gdGhlIGJhZyBhbmQgbG9vayBhdCB3aGF0IHdlIGdldAoKIVtdKGltYWdlcy9waWNrZWQtYmFsbHMucG5nKQoKCgpXZSBjYW4gZXhwcmVzcyBhIHBhcnRpY3VsYXIgY2hvaWNlIG9mIGJhbGxzIGFzIGEgdGFibGU6LQoKfCAgfCBJbiBTZWxlY3Rpb24gfCBOb3QgSW4gU2VsZWN0aW9uIHwgVG90YWwKLS0tLS0tLS0tLS0tLSB8IC0tLS0tLS0tLS0tLS0gfCAgLS0tLS0tLS0tLS0tLSB8IC0tLS0tLS0tLS0tLS0gfCAKR3JleSBTZWxlY3RlZCB8IDggfCAyIHwgMTAgfApHcmV5IE5vdCBTZWxlY3RlZCAgfCA0IHwgMjYgfCAzMCB8ClRvdGFsIHwgMTIgfCA2IHwgNDAgfAoKCgpUaGUgZm9ybXVsYSBmb3IgRmlzaGVycyBleGFjdCB0ZXN0IGlzOwoKJCQgcCA9IFxmcmFje1xiaW5vbXthICsgYn17YX1cYmlub217YyArZH17Y319e1xiaW5vbXtufXthICtjfX0gPSBcZnJhY3soYStiKSEoYytkKSEoYStjKSEoYitkKSF9e2EhYiFjIWQhbiF9ICQkCndpdGggOi0KCnwgIHwgSW4gU2VsZWN0aW9uIHwgTm90IEluIFNlbGVjdGlvbiB8IFRvdGFsCi0tLS0tLS0tLS0tLS0gfCAtLS0tLS0tLS0tLS0tIHwgIC0tLS0tLS0tLS0tLS0gfCAtLS0tLS0tLS0tLS0tIHwgCkdyZXkgU2VsZWN0ZWQgfCBhIHwgYiB8IGEgICsgIGIgfApHcmV5IE5vdCBTZWxlY3RlZCAgfCBjIHwgZCB8IGMgKyBkIHwKVG90YWwgfCBhICsgYyB8IGIgK2QgfCBhICsgYiArIGMgKyBkICg9bikgfAoKCgoKb3IgbGVzcyBmb3JtYWxseTsKCipQID0gKHdheXMgb2YgY2hvb3NpbmcgZ3JleSBiYWxscykgWCAod2F5cyBvZiBub24tZ3JleSBiYWxscyBhbW9uZ3N0IHN1YnNldCkgLyB3YXlzIG9mIGNob29zaW5nIHN1YnNldCoKCgpGb3J0dW5hdGVseSwgd2UgaGF2ZSBzb2Z0d2FyZSB0byBjYWxjdWxhdGUgdGhlc2UgcXVhbnRpdGllcyEKCiMjIFNvZnR3YXJlIGZvciBjb25kdWN0aW5nIGEgb3Zlci1yZXByZXNlbnRhdGlvbiB0ZXN0IChnb3NlcSkKCldlIHdpbGwgYmUgdXNpbmcgW2Bnb3NlcWBdKGh0dHBzOi8vd3d3LmJpb2NvbmR1Y3Rvci5vcmcvcGFja2FnZXMvcmVsZWFzZS9iaW9jL2h0bWwvZ29zZXEuaHRtbCksIHdoaWNoIGlzIGEgc29mdHdhcmUgcGFja2FnZSBhdmFpbGFibGUgdGhyb3VnaCBbQmlvY29uZHVjdG9yXSh3d3cuYmlvY29uZHVjdG9yLm9yZy8pLiBIb3dldmVyLCByYXRoZXIgaGF2aW5nIHRvIHdyaXRlIFIgY29kZSwgd2Ugd2lsbCBiZSB1c2luZyB0aGUgcGFja2FnZSB0aHJvdWdoIG91ciBpbnN0aXR1dGUncyBHYWxheHkgc2VydmVyLgoKVGhpcyBwYWNrYWdlIGhhcyBiZWVuICpzcGVjaWZpY2FsbHktZGV2ZWxvcGVkKiBmb3IgdXNlIHdpdGggUk5BLXNlcSBkYXRhLiBQbGVudHkgb2YgbWV0aG9kcyBoYXZlIGJlZW4gYXBwbGllZCB0byBtaWNyb2FycmF5IGRhdGEsIGJ1dCB0aGUgYXNzdW1wdGlvbnMgbWlnaHQgbm90IGhvbGQgZm9yIFJOQS1zZXEuIEZvciBpbnN0YW5jZSwgYGdvc2VxYCB3aWxsIGFkanVzdCBmb3IgdGhlIGxlbmd0aCBvZiBhIGdlbmUuCgojIyBQcmVwYXJpbmcgdGhlIGRhdGEgZm9yIGFuIG92ZXItcmVwcmVzZW50YXRpb24gdGVzdAoKV2UgY2FuIHN0YXJ0IHdpdGggdGhlIGxpc3QgdGhhdCBjb250YWlucyByZXN1bHRzIGZvciAqYWxsKiBnZW5lczsgTGV0J3MgdGFrZSBgdDQ3ZF9UcmVhdG1lbnRfREVBX1Byb2ctdnMtQ29udHJvbF9hbGwuY3N2YAoKYGBge3J9CmRlVGFibGUgPC0gcmVhZC5jc3YoInQ0N2RfVHJlYXRtZW50X0RFQV9Qcm9nLXZzLUNvbnRyb2xfYWxsLmNzdiIpCmRlVGFibGUKYGBgCgpUaGUgZ29hbCBpcyB0byBvYnRhaW4gYSB0YWJsZSB3aXRoIHR3byBjb2x1bW5zOyB0aGUgZmlyc3QgY29udGFpbmluZyB0aGUgZ2VuZSBpZGVudGlmaWVyLCBhbmQgdGhlIHNlY29uZCBiZWluZyBgVFJVRWAgb3IgYEZBTFNFYCBpbmRpY2F0aW5nIHdoZXRoZXIgdGhlIGdlbmUgaXMgZGlmZmVyZW50aWFsbHktZXhwcmVzc2VkIGF0IGEgZ2l2ZW4gY3V0LW9mZiAoZS5nLiBgMC4wNWApLiAKClRoZXJlIGFyZSBzZXZlcmFsIHdheXMgb2YgZG9pbmcgdGhpcy4gSWYgeW91IHdpc2ggdG8gc2VlIHRoZSBSIGNvZGUgKHVzaW5nIGBkcGx5cmApLCBjbGljayB0aGUgJ0NvZGUnIGJ1dHRvbiBvbiB0aGUgcmlnaHQuCgpgYGB7ciBtZXNzYWdlPUZBTFNFfQpsaWJyYXJ5KGRwbHlyKQpkZVRhYmxlIDwtIHJlYWQuY3N2KCJ0NDdkX1RyZWF0bWVudF9ERUFfUHJvZy12cy1Db250cm9sX2FsbC5jc3YiKQptdXRhdGUoZGVUYWJsZSwgREUgPSBwYWRqIDwgMC4wNSkgJT4lIAogIG11dGF0ZShERSA9IGlmZWxzZShpcy5uYShERSksRkFMU0UsREUpKSAlPiUgCiAgc2VsZWN0KFgsIERFKSAtPiBmaWx0ZXJlZFRhYmxlCiAgd3JpdGUudGFibGUoZmlsdGVyZWRUYWJsZSwiZGUtdGFibGUtZm9yLWdvc2VxLnR4dCIscXVvdGU9RkFMU0Uscm93Lm5hbWVzPUZBTFNFKQpmaWx0ZXJlZFRhYmxlCmBgYAoKTWFuaXB1bGF0aW5nIHRoZSBmaWxlIGNhbiBhbHNvIGJlIGRvbmUgaW4gR2FsYXh5LgoKIyMgSW1wb3J0aW5nIHRoZSBHZW5lIGxpc3QgaW50byBHYWxheHkKCiMjIyBJbXBvcnQgdGhlIGNzdiBmaWxlCgoqKipHZXQgRGF0YSoqKiAtPiAqKipVcGxvYWQgRGF0YSoqKgoKIVtdKGltYWdlcy9nYWxheHktMS5wbmcpCgojIyMgQ29udmVydCB0byB0YWJ1bGFyIGZvcm0KCioqKlRleHQgTWFuaXB1bGF0aW9uKioqIC0+ICoqKkNvbnZlcnQgZGVsaW1pdGVycyB0byBUQUIqKioKClNlbGVjdCAqQ29tbWFzKiBmcm9tIGRyb3AtZG93biBtZW51CgohW10oaW1hZ2VzL2dhbGF4eS0yLnBuZykKCiMjIyBSZW1vdmUgaGVhZGVyIGxpbmUKCioqKlRleHQgTWFuaXB1bGF0aW9uKioqIC0+ICoqKlNlbGVjdCBsYXN0IGxpbmVzIGZyb20gYSBkYXRhc2V0KioqCgpNYWtlIG5vdGUgb2YgaG93IG1hbnkgbGluZXMgaW4gZmlsZSwgKk4qIGFuZCByZW1vdmUgKk4tMSogbGFzdCBsaW5lcyB3aXRoIHRoaXMgdG9vbAohW10oaW1hZ2VzL2dhbGF4eS0zLnBuZykKCiMjIyBSZXRhaW4gY29sdW1ucyBvZiBpbnRlcmVzdAoKUHV0ICpjMSxjMixjMyxjNCxjNSxjNixjNyxjMTIqIHRvIHJldGFpbiBjb2x1bW5zIDEsMiwzLDQsNSw2LDcgYW5kIDEyCgoqKipUZXh0IE1hbmlwdWxhdGlvbioqKiAtPiAqKipDdXQgY29sdW1ucyBmcm9tIGEgdGFibGUqKioKCiFbXShpbWFnZXMvZ2FsYXh5LTQucG5nKQoKIyMjIFRoZSBjbGVhbmVkIGRhdGFzZXQKClRoZSBvdXRwdXQgZnJvbSB0aGUgcHJldmlvdXMgc3RlcCB3aWxsIGJlIHJlZmVycmVkIHRvIGFzIG91ciAqY2xlYW5lZCBkYXRhc2V0KgoKIVtdKGltYWdlcy9nYWxheHktNS5wbmcpCgoKIyMgUGVyZm9ybWluZyBhIEdlbmUgU2V0IEFuYWx5c2lzIGluIEdhbGF4eQoKRnJvbSBvdXIgY2xlYW5lZCB0YWJsZSBpbiB0aGUgcHJldmlvdXMgc3RlcHMsIHdlIG5lZWQgdG8gY29tcHV0ZSBpZiBlYWNoIGdlbmUgaXMgREUgYXQgMC4wNSBzaWduaWZpY2FuY2UgbGV2ZWwKCiMjIyBBZGRpbmcgYW4gZXh0cmEgY29sdW1uCgoqKipUZXh0IE1hbmlwdWxhdGlvbioqKiAtPiAqKipDb21wdXRlIGFuIGV4cHJlc3Npb24gb24gZXZlcnkgcm93KioqCgpVc2UgdGhlIGV4cHJlc3Npb24gYGM3PDAuMDVgIHRvIHRlc3QgaWYgdGhlIGFkanVzdGVkIHAtdmFsdWUgaXMgbGVzcyB0aGFuIDAuMDUKCiFbXShpbWFnZXMvZ2FsYXh5LTYucG5nKQoKV2hpY2ggc2hvdWxkIGdpdmUgc29tZXRoaW5nIGxpa2UgdGhpcwoKIVtdKGltYWdlcy9nYWxheHktNy5wbmcpCgojIyMgRXh0cmFjdGluZyB0aGUgY29sdW1ucyBuZWVkZWQgZm9yIGdvc2VxCgoqKipUZXh0IE1hbmlwdWxhdGlvbioqKiAtPiAqKipDdXQgY29sdW1ucyBmcm9tIGEgdGFibGUqKioKQ2hvb3NlIGBjMSxjOWAgYXMgdGhlIGNvbHVtbnMgdG8gY3V0CgohW10oaW1hZ2VzL2dhbGF4eS04LnBuZykKCldoaWNoIHNob3VsZCBnaXZlIHRoZSBmb2xsb3dpbmcKCiFbXShpbWFnZXMvZ2FsYXh5LTkucG5nKQoKIyMjIFJ1bm5pbmcgZ29zZXEKCioqKk5HUzpSTkEtc2VxKiogLT4gZ29zZXEKClNlbGVjdCBHZW5vbWUgdmVyc2lvbiAqaGcxOSogaW4gdGhpcyBjYXNlLiBDb3VsZCBhbHNvIHNwZWNpZnkgd2hpY2ggdHlwZSBvZiBpZGVudGlmaWVycyBjYW4gYmUgZm91bmQgaW4gdGhlIGZpcnN0IGNvbHVtbi4gKkVuc2VtYmwqIElEIGlzIGNvcnJlY3QgaW4gdGhpcyBjYXNlLgoKIVtdKGltYWdlcy9nYWxheHktMTAucG5nKQoKCiMjIFRoZW9yeSBQYXJ0IElJOiBUaHJlc2hvbGQtZnJlZQoKRm9yIHRoZXNlIHRlc3RzLCB3ZSBkb24ndCBoYXZlIHRvIHNwZWNpZnkgYSBzdGF0aXN0aWNhbCB0aHJlc2hvbGQgYW5kIHVzZSB0aGUgdGVzdCBzdGF0aXN0aWNzIGZyb20gKmFsbCogZ2VuZXMgYXMgdGhlIGlucHV0IHRvIHRoZSB0ZXN0LiBUaGUgcG9wdWxhciAqR2VuZSBTZXQgRW5yaWNobWVudCBBbmFseXNpcyAoR1NFQSkqIHVzZXMgdGhpcyBhcHByb2FjaC4gVGhlc2UgdGVzdHMgY2FuIGJlIHVzZWQgdG8gZGV0ZWN0IGRpZmZlcmVudGlhbCBleHByZXNzaW9uIGZvciBhIGdyb3VwIG9mIGdlbmVzLCBldmVuIHdoZW4gdGhlIGVmZmVjdHMgYXJlIHRvbyBzbWFsbCBvciB0aGVyZSBpcyB0b28gbGl0dGxlIGRhdGEgdG8gZGV0ZWN0IHRoZSBnZW5lcyBpbmRpdmlkdWFsbHkuCgohWypTdWJyYW1hbmlhbiBldCBhbCwgUE5BUyAyMDA1Kl0oaW1hZ2VzL0dTRUEucG5nKQoKLSBSYW5rIGFsbCB5b3VyIGdlbmVzIGFjY29yZGluZyB0byB0ZXN0IHN0YXRpc3RpYyBvciBmb2xkLWNoYW5nZQotIEZvciBhIHBhcnRpY3VsYXIgZ2VuZSBzZXQgb2YgaW50ZXJlc3QsIGxvb2sgZG93biB0aGUgZ2VuZSBsaXN0IGFuZCBjb21wdXRlIGEgc2NvcmUKICAgICsgR28gdXAgb25lIHVuaXQgb24gdGhlIHktYXhpcyBpZiB0aGUgZ2VuZSB5b3UgZW5jb3VudGVyIGlzIGNvbnRhaW5lZCBpbiB5b3VyIGdlbmUgbGlzdAogICAgKyBHbyBkb3duIG9uZSB1bml0IGlmIHRoZSBnZW5lIGlzIG5vdCBjb250YWluZWQgaW4gdGhlIGdlbmUgbGlzdAogICAgKyBQbG90IHRoZSBzY29yZSBhZ2FpbnN0IHBvc2l0aW9uIGluIGdlbmUgbGlzdAogICAgKyBJZiB0aGVyZSBpcyBhIHBlYWsgYXQgdGhlIGxlZnQgb3IgcmlnaHQsIHRoZW4geW91ciBzZXQgaXMgZW5yaWNoZWQuCgoKVGhlIEJyb2FkIGluc3RpdHV0ZSBwcm92aWRlcyBbYSB2ZXJzaW9uIG9mIEdTRUFdKGh0dHA6Ly9zb2Z0d2FyZS5icm9hZGluc3RpdHV0ZS5vcmcvZ3NlYS9pbmRleC5qc3ApIHRoYXQgY2FuIGJlIHJ1biB2aWEgYSBqYXZhIGFwcGxpY2F0aW9uLiAKVGhlIHJlcXVpcmVkIGlucHV0IHRleHQgY29udGFpbnMgYSBzZXQgb2YgZ2VuZSBpZGVudGlmaWVycyB0aGF0IGFyZSBvcmRlcmVkIGFjY29yZGluZyB0byBzb21lIHRlc3Qgc3RhdGlzdGljLgoKYGBge3J9CmxpYnJhcnkoZHBseXIpCmRlVGFibGUgPC0gcmVhZC5jc3YoImRlc2VxMl9yZXN1bHRzL3Q0N2RfVHJlYXRtZW50X0RFQV9Qcm9nLXZzLUNvbnRyb2xfYWxsLmNzdiIpCmRlVGFibGUgJT4lIGFycmFuZ2UoZGVzYyhzdGF0KSkgJT4lIAogIGRwbHlyOjo6c2VsZWN0KHN5bWJvbCwgc3RhdCkgJT4lIAogIGZpbHRlcighaXMubmEoc3ltYm9sKSkgLT4gb3JkZXJlZFRhYmxlCm9yZGVyZWRUYWJsZQpgYGAKCgoKIyMgUHJlcGFyaW5nIHRoZSBnZW5lIHNldCBmb3IgYSBHU0VBIGFuYWx5c2lzCgojIyMgUmFua2luZyB0aGUgZ2VuZXMKCkZyb20gdGhlICpjbGVhbmVkKiB0YWJsZSBjcmVhdGVkIGFib3ZlLCBzb3J0IG9uIGNvbHVtbiA1ICh0aGUgdGVzdCBzdGF0aXN0aWMpCgoqKipGaWx0ZXIgYW5kIFNvcnQqKiogLT4gKioqU29ydCBkYXRhIGluIGFzY2VuZGluZyBvciBkZXNjZW5kaW5nIG9yZGVyKioqCgohW10oaW1hZ2VzL2dhbGF4eS0xMC5wbmcpCgpHaXZpbmcgdGhlIG91dHB1dDotCgohW10oaW1hZ2VzL2dhbGF4eS0xMS5wbmcpCgojIyMgQ3V0dGluZyB0aGUgY29sdW1ucwoKRXh0cmFjdCBjb2x1bW5zIDggKGdlbmUgc3ltYm9sKSBmb2xsb3dlZCBieSB0ZXN0IHN0YXRpc3RpYzsgYGM4LGM1YAoKKioqVGV4dCBNYW5pcHVsYXRpb24qKiogLT4gKioqQ3V0IGNvbHVtbnMgZnJvbSBhIHRhYmxlKioqCgohW10oaW1hZ2VzL2dhbGF4eS0xMi5wbmcpCgoKIyMgUnVubmluZyB0aGUgR1NFQVByZXJhbmtlZCB0b29sCgpJbiBvdXIgR2FsYXh5IHNlcnZlciB3ZSBwcm92aWRlIGFuIGludGVyZmFjZSB0b2BmZ3NlYWAsIHdoaWNoIGlzIGEgQmlvY29uZHVjdG9yIHBhY2thZ2UgZm9yIHBlcmZvcm1pbmcgdGhlIEdTRUEgYWxnb3JpdGhtIG9uIGEgcHJlLXJhbmtlZCBnZW5lIGxpc3QuCgpXZSBjYW4gZ2l2ZSBpdCB0aGUgdGFibGUgd2UgaGF2ZSBnZW5lcmF0ZWQgaW4gdGhlIHByZXZpb3VzIHN0ZXAKCgpHaXZlcyBhbGwgcmVzdWx0cyByYW5rZWQgYWNjb3JkaW5nIHRvLi4KCi0gQ2FuIHdlIGlkZW50aWZ5IGdlbmUgc2V0cyB0aGF0IGFyZSBzaWduaWZpY2FudCBhdCAwLjA1PwotIFdoYXQgZ2VuZXMgYXJlIG92ZXItIGFuZCB1bmRlci1yZXByZXNlbnRlZD8KLSBDYW4geW91IHJ1biB0aGUgdG9vbCBvbiBhbm90aGVyIGRhdGFzZXQ/